SDSC’s Gordon Supercomputer Assists in Crunching Large Hadron Collider Data

Published 04/04/2013

UC San Diego Physics Professor Frank Wuerthwein. Photo: Ben Tolo/SDSC

Gordon, the unique supercomputer launched last year by the San Diego Supercomputer Center (SDSC) at the University of California, San Diego, recently completed its most data-intensive task so far: rapidly processing raw data from almost one billion particle collisions as part of a project to help define the future research agenda for the Large Hadron Collider (LHC). 

Under a partnership between a team of UC San Diego physicists and the Open Science Grid (OSG), a multi-disciplinary research partnership funded by the U.S. Department of Energy and the National Science Foundation, Gordon has been providing auxiliary computing capacity by processing massive data sets generated by the Compact Muon Solenoid, or CMS, one of two large general-purpose particle detectors at the LHC used by researchers to find the elusive Higgs particle. 

“This exciting project has been the single most data-intensive exercise yet for Gordon since we completed large-scale acceptance testing back in early 2012,” said SDSC Director Michael Norman, who is also an astrophysicist involved in research studying the origins of the universe. “I’m pleased that we were able to make Gordon’s capabilities available under this partnership between UC San Diego, the OSG, and the CMS project.” 

The around-the-clock data processing run on Gordon was completed in about four weeks’ time, making the data available for analysis several months ahead of schedule. About 1.7 million core hours – or about 15% of Gordon’s total compute capacity - were dedicated to this task, with more than 125 terabytes of data streaming through Gordon’s nodes and into SDSC’s Data Oasis storage system for further analysis. Just one terabyte of data, or one trillion bytes, equals the information printed on paper made from 50,000 trees. 

“Access to Gordon, and its excellent computing speed due to its flash-based memory, really helped push forward the processing schedule for us,” said Frank Wuerthwein, a professor of physics at UC San Diego and a member of the CMS project. “With only a few weeks’ notice, we were able to gain access to Gordon and complete the runs, making the data available for analysis in time to provide crucial input toward international planning meetings on the future of particle physics.”   

“Giving us access to the Gordon supercomputer effectively doubled the data processing compute power available to us,” added Lothar Bauerdick, OSG’s executive director and the U.S. software and computing manager for the CMS project. “This gives CMS scientists precious months to get to their science analysis of the data reconstructed at SDSC.”

The UC San Diego-OSG collaboration comes as the LHC was shut down in February 2013 to make numerous upgrades during the next two years. One major activity during the shutdown includes the development of plans for efficient, effective searches once the LHC is back in operation. To do that – and to have time enough to upgrade equipment – researchers must also sift through massive amounts of stockpiled data to help define future research agendas. 

“Unfortunately, the shutdown schedule meant that the parked data would not be available for analysis this summer, and possibly not even for deriving meaningful contributions to planning documents for future upgrades of the experiment that are due this fall,” explained Wuerthwein. 

This image of a supersymmetry event shows the transverse momentum imbalance due to dark matter particles escaping the detector (direction indicated by red arrow). Red and blue rectangles indicate energy deposited in the electromagnetic and hadronic calorimeter respectively; green tracks in the center show charged particles with transverse momentum larger than 2GeV. Yellow-outlined triangles indicate jet cones or the presence of subatomic particles called quarks. Image courtesy of Matevz Tadel, UC San Diego/CMS [view large image]

The Hunt for Dark Matter

With the recent discovery and later confirmation in March of the Higgs boson – the last missing piece of the standard model of particle physics – scientists are now setting their sights on discovering new physics beyond the standard model. The next big thing is to search for dark matter, according to Wuerthwein.  

“For the Higgs, we knew exactly how to search for it given theoretical predictions based on past experimental results,” said Wuerthwein, who is heading up the search for dark matter for the entire CMS team. “For dark matter, the situation is much more hazy. We hope to produce dark matter at the LHC in cascade decays of a whole spectrum of new fundamental particles, the lowest mass of which is dark matter. But the details of this spectrum of masses are unknown. To have sensitivity to a larger range of possible mass spectra, we needed to write more data to tape so we would be able to carefully analyze it later.” 

The origin of this spectrum of new fundamental particles is a new kind of symmetry of nature called Supersymmetry, or SUSY. “Underlying this symmetry is a fascinating but theoretical conjecture with little to no physical evidence so far,” noted Wuerthwein. “It's fascinating because it could provide an ordering principle that allows for all known physical forces to be unified during the earliest times of the ‘Big Bang’ or birth of the universe, while providing an explanation for dark matter, and resolving some of the outstanding questions about details of the Higgs mechanism and mass.” 

UC San Diego researchers and CMS team members, in addition to Wuerthwein, include Jim Branson, Vivek Sharma, and Avi Yagil, all of whom played major roles in the discovery of the Higgs particle and will continue to make meaningful contributions to future related research.

“UC San Diego has been one of the most successful institutions in the global hunt for the Higgs particle discovery at the LHC,” said Wuerthwein, who is leading the university’s search for dark matter.  

Wuerthwein and his colleagues will present additional details of the CMS collaboration at the 20th International Conference on Computing in High Energy and Nuclear Physics (CHEP) to be in Amsterdam, The Netherlands October 14-18, 2013. More about UC San Diego’s role in the quest to find the Higgs particle can be found online. 

About SDSC
As an Organized Research Unit of UC San Diego, SDSC is considered a leader in data-intensive computing and cyberinfrastructure, providing resources, services, and expertise to the national research community, including industry and academia. Cyberinfrastructure refers to an accessible, integrated network of computer-based resources and expertise, focused on accelerating scientific inquiry and discovery. SDSC supports hundreds of multidisciplinary programs spanning a wide variety of domains, from earth sciences and biology to astrophysics, bioinformatics, and health IT. With its two newest supercomputer systems, Trestles and Gordon, SDSC is a partner in XSEDE (Extreme Science and Engineering Discovery Environment), the most advanced collection of integrated digital resources and services in the world.  

About the Open Science Grid
The Open Science Grid (OSG) is a multi-disciplinary partnership to federate local, regional, community and national cyberinfrastructures to meet the needs of research and academic communities at all scales.  Jointly funded by the U.S. Department of Energy and the National Science Foundation, OSG provides provide common service and support for resource providers and scientific institutions using a distributed fabric of high throughput computational services. The OSG does not own resources but provides software and services to users and resource providers alike to enable the opportunistic usage and sharing of resources.

Media Contacts:
Jan Zverina, SDSC Communications
858 534-5111 or jzverina@sdsc.edu

Warren R. Froelich, SDSC Communications
858 822-3622 or froelich@sdsc.edu

Ruth Pordes, OSG Communications and Council Chair
630 840-3921 or ruth@fnal.gov

Related Links:

San Diego Supercomputer Center: http://www.sdsc.edu/
UC San Diego Department of Physics: http://www-physics.ucsd.edu/
UC San Diego: http://www.ucsd.edu/
Open Science Grid: https://www.opensciencegrid.org/bin/view